Anchoring Bias in Online Voting 
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Voting online with explicit ratings could largely reflect people's preferences and objects' qualities, 
but ratings are always irrational, because they may be affected by many unpredictable factors like 
mood, weather, as well as other people's votes. By analyzing two real systems, this paper reveals a 
systematic bias embedding in the individual decision-making processes, namely people tend to give a 
low rating after a low rating, as well as a high rating following a high rating. This so-called anchoring 
bias is validated via extensive comparisons with null models, and numerically speaking, the extent 
of bias decays with interval voting number in a logarithmic form. Our findings could be applied 
in the design of recommender systems and considered as important complementary materials to 
previous knowledge about anchoring effects on financial trades, performance judgements, auctions, 
and so on. 

PACS numbers: 89.20.Hh, 89.20.Ff, 89.65.-s, 89.75.Fb 



Uncovering human behavioral patterns, such as bursty 
nature of temporal activity [l|, Q , scaling laws of human 
travel [1, H , different selecting patterns of different kinds 
of users [sl, onto different kinds of objects 0, is 
significant to understand many socioeconomic phenom- 
ena and provide high-quality services. Here we investi- 
gate online voting, which contains huge business value 
in e-commerce. Take recommender systems as an exam- 
ple, via analyzing online votes, they can automatically 
find out suitable products for every customer Q. So it 
could largely improve the performance of recommender 
systems, if we make clear the knowledge about how peo- 
ple vote [ISIiii. 

In some systems, votes are confined to only two 
extremes-like or dislike, while in some other systems, 
people can vote with explicit ratings-usually from one 
star to five stars. Explicit ratings, however, do NOT sig- 
nify rational judgments. Indeed, people's votes may be 
largely affected by prior votes If 211 and social pressure 
(like suggestions from friends) We do not consider 

the aforementioned biases in this paper, in that when 
one votes on the systems we analyze here, neither others' 
votes nor social network services are provided for users, 
but by comparing with null models, considerable vot- 
ing bias arc still observed, which originates from internal 
decision-making processes of individuals, that is to say, 
people strongly tend to give a low rating again after vot- 
ing on a low-quality object, as well as to give a high rating 
again after voting on a high-quality object. We name it 
as anchoring bias, since it is similar to the well-known 
anchoring effects in purchases [T5[ . auctions [l6l . Il7| , 
judgements [islligl and estimations |20l.[2lj (see also the 
review articl e 1221 and the references therein). Previous 
experiments j22| showed that even a randomly assigned 
initial value of an object could remarkably affect our es- 
timation on its real value. This paper indicates that a 
prior vote on another object could affect our current vote 



because we may take that prior vote as an anchor. 



TABLE I: Basic statistics of MovieLens and WikiLens. A*', 
M and V denote the number of users, objects and ratings, 
respectively, p — denotes the sparsity of the data, and 
(r) is the average rating over all votes. 



Data Set N 


M 


V 


P 


(r) 


MovieLens 6040 


3952 


1000292 


0.042 


3.58 


WikiLens 289 


4951 


26937 


0.019 


3.71 



In this paper, we consider two real data sets, Movie- 
Lens and WikiLens. MovieLens is a movie rating sys- 
tems with five stars (i.e., ratings can be 1, 2, 3, 4 
and 5). The WikiLens is a generalized collaborative 
recommender system that allows its community to de- 
fine object types (e.g., beer) and categories (e.g., micro- 
brews), and vote on objects. Ratings in WikiLens can 
be 1, 1.5, • • ■ , 4.5, 5. Both the two data sets can be found 



in GroupLens research web (http://gr0uplens.0rg/l), and 



their basic statistics are summarized in Table I. 

A recommender system with explicit ratings can be de- 
scribed by a weighted bipartite network where each vote 
is represented by an edge connecting the corresponding 
user and object, and its weight is defined as the corre- 
sponding rating. The degree of a user is defined as the 
number of objects she has voted, while the degree of an 
object is the number of users who have voted on it. Fig- 
ure 1 reports the degree distributions for users, which do 
not follow neither the power-law form nor the exponen- 
tial form. In fact, they lie in between exponential and 
power-law forms, and can be well fitted by the so-called 
stretched exponential distributions 
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where K is a constant and c (0 < c < 1) is the character- 
istic exponent. The borderline c = 1 corresponds to the 
usual exponential distribution. For c smaller than 1, the 
distribution presents a clear curvature in a log-log plot. 
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FIG. 1: Distributions of user degrees, which obey the 
stretched exponential form. We therefore plot the cumula- 
tive distribution P{ku) instead of p{ku) and show the linear 
fittings of ln(— InP(fcu)) vs. Infe^ in the insets. 



The exponent c can be determined by considering the 
cumulative distribution 



P{k) ^ exp 
which can be rewritten as 



ln(-lnP(fc)) - clnfc. 



(2) 



(3) 



Therefore, using Ink as x-axis and ln(— InP(fc)) as y- 
axis, if the corresponding curve can be well fitted by a 
straight line, then the slope equals c. Accordingly, as 
shown in figure 1, the exponents c for MovieLens and 
WikiLens arc 0.92 and 0.53, respectively. Note that, 
the user degree distribution of MovieLens is very close 
to a usual exponential form. This kind of distributions 
often displays a mixture of power-law and exponential 
form |25(] , and are usually fitted by stretched exponential 
function [2^, power law with exponential cutoff [2^ or 
Mandelbrot law (g^j . Often, the head is closer to a power 
law while the tail shows exponential decay due to the lim- 
itation of people's ability in accessing information. Since 
the MovieLens data only consists of users having voted on 
no less than 20 movies, the head part cannot be observed 
and thus the distribution is close to an exponential form. 
Figure 2 reports the object degree distributions that also 
obey the stretched exponential form. Note that, in some 
other online user-object bipartite networks (e.g., audio- 
scrobblcr.com and dclicious.com) where the number of 
users is huge and users are not required to vote on ob- 
jects, the object degree can be very well characterized by 
power-law distribution 0, . The distinct statistics of 
object-degree distributions of the present systems have 
refined our knowledge about online user-object bipartite 
networks and raised open question about whether the 
huge number of users and/or less efforts of actions are 
necessary to the appearance of power laws. 

To demonstrate the presence of anchoring bias, wc first 
look at an extreme case: Will we vote with systematic 
bias after voting on some very high-quality or very low- 
quality object~we name these objects as outliers. In the 
absence of systematic bias, the next votes after outliers' 
votes should be more or less the same to usual votes; 
while if the anchoring bias exists, a vote on an outlier will 
become the anchor of the next vote, and thus in average 



FIG. 2: Distributions of object degrees, which also obey the 
stretched exponential form. 



wc will give high rating after voting on a high-quality 
object and low rating after a low-quality object. 



TABLE II: Basic statistics of outliers. The six columns from 
the second to the last one are the number of low-quality out- 
liers (#LQO), the number of votes right after votes on low- 
quality outliers (#A~), the number of high-quality outliers 
(#HQO), the number of votes right after votes on high-quality 
outliers (#A^), the number of votes after votes on outliers 
(#A~&A"'"), and the percentage of these after-outlier votes 
in all votes. 

Data Set #LQO #A- #HQO #A+ ij^A- kA+ Percentage 



MovieLens 
WikiLens 



97 
12 



8526 
268 



14 
13 



10713 
370 



19239 
638 



1.92% 
2.37% 



We use the average rating to estimate an object's qual- 
ity, and to reduce the possible errors caused by person- 
alized tastes and unreasonable votes, we only consider 
the objects getting more than ten votes. Although rat- 
ings cannot perfectly refiect qualities, they are correlated 
with qualities and can be naturally treated as anchors 
by users. For both MovieLens and WikiLens, an object 
(with more than ten votes) is distinguished as low-quality 
or high-quality outlier if its average rating is lower than 
2.0 or higher than 4.5. 

Denote by r^Q, the rating from user i onto object 
a, and for an arbitrary user i, all her ki ratings 
are ordered by time as ViOi -i ^i02 1 "^iOi , I'iOk i where 
Oi, O2, O3, • • • , Oki are the objects having been voted by 
i, Tio^ is the oldest rating, and r^o^ is the most recent 
rating. If Oi(l < ki) is a low-quality outlier, riO;+i 
an after- low-quality-outlier rating (A" rating for short), 
while if Oi is a high-quality outlier. ViOi^-^ rating. 
According to the above criterion and definition, as shown 
in Table II, there are in total 19239 (1.92% of aU ratings) 
after-outlier ratings for MovieLens and 638 (2.37% of all 
ratings) after-outlier ratings for WikiLens. One could 
observe that the high-quality outliers get more votes in 
average, which is in accordance with our common sense 
that better objects are more popular. 

We next compare the votes after low-quality outliers 
and those after high-quality outliers, namely A~ and 
votes. The average rating among all A~ votes is defined 
as 



1 



1^- 



E 



(4) 



TABLE III: Statistics of votes after outliers for MovieLens. 
(r), (do) and (d„) respectively denote the average rating, the 
average difference to object average and the average difference 
to user average. 



MovieLens 


(r) 


(do) 




A~ votes 


2.72 


-0.054 


-0.635 


votes 


4.16 


0.033 


0.449 



TABLE IV: Statistics of votes after outliers for WikiLens. 
(r), {do} and (du) respectively denote the average rating, the 
average difference to object average and the average difference 
to user average. 



WikiLens 


(r) 


{do) 


(du) 


A~ votes 


2.63 


-0.071 


-0.850 


A^ votes 


4.13 


0.061 


0.372 



3 




mnii [111 I III I 

20 40 60 80 100 

\/ot/ng No. (b) 

FIG. 3: (Color online) Rating series of a typical user in Movie- 
Lens who has voted 106 movies in total. All these 106 ratings 
are displayed according to the voting time in panel (a), and 
the positive and negative ratings are respectively represented 
by light-green and dark-green lines in panel (b). 



In addition, we look at the difference between a rating 
ria £ A" and the average rating on the object a, as well 
as the difference between r^a and the average rating by 
the user i. Accordingly, we define the average difference 
to object average as 



(5) 



where {r,a) denotes the average rating on a, and the 
average difference to user average as 



(du) 



1 



(6) 



where (r^,) denotes the average rating by i. Analogously, 
we can define (r+), (d^) and (d^) for votes. 

Table III and Table IV show the remarkable difference 
between people's votes after low-quality and high-quality 
outliers, respectively. The results indicate the possible 
presence of anchoring bias, that is, people tend to give a 
low rating if the prior- visited object is not good, and vice 
versa. However, the above evidence is not solid enough 
since it covers only a tiny fraction of votes, and thus we 
will further analyze the full rating series of every user. 

To get rid of the effects of different voting standards 
of users (e.g., some users are good-tempered and tend to 
give high ratings than others) and different deserving rat- 
ings of objects (e.g., some objects are of better qualities 
and should be voted with high ratings), we regulate each 
rating in the following four ways: (i) to eliminate the 
average rating over all votes as = — (^}; (h) to 
eliminate the average rating over all votes on the corre- 
sponding object as r'^^ = ria — {fma)', (hi) to eliminate the 
average rating over all votes on the corresponding user as 
''ict ^ ''^ia ^ (iv) to eliminate both average ratings 

as r'ia = [ria - {r,a)) + (na - (ri*))- Readers are easy to 
reproduce all the following experiments and will find that 
the four cases lead to qualitatively the same results, and 
thus we only present the results of case (iv) hereinafter 



and without specific statement, the term rating(s) stands 
for regulated rating(s) of case (iv). 

Figure 3(a) presents the rating series of a typical user 
in MovieLens. We simply divide ratings into two classes- 
positive ratings and negative ratings, and display them 
without explicit values in figure 3(b), where one could 
observe that ratings in the same class are aggregated. 
This kind of aggregation reveals the anchoring bias in 
voting behavior, namely people is likely to give a high 
rating after a prior high rating while is likely to give a 
low rating after a prior low rating. Similar to the method 
used to measure the memory effect of a time series p9| . 
to quantify the aggregation phenomenon for an arbitrary 
user i, we calculate the Pearson correlation coefficient 
i?j(-l < Ri <1) of two series riOi i '^iOa ; ' • • i'"iOfc,_i and 
fi02 1 "i^iOa J • • • j^iOfc . , where the Pearson correlation coeffi- 
cient for two finite series xi,X2, • • ■ ,Xn and yi , ?/2 , ■ " ' ; 2/n 
is defined as 



R(x,y) 



VE:Li(^.-(^))VEr=i(y.-(2/»^ 



(7) 



According to the definition, a positive Ri indicates that 
the user i may have the anchoring bias. 

We compare the empirical results with those of a null 
model, in which each user's voting times are randomly 
redistributed. That is to say, for an arbitrary user i, the 
rating series {riOi, '^i02, •••j'^iOfc, }, is reordered. Fig- 
ure 4 compares the distributions of Pearson correlation 
coefficients R of the empirical data and the null model. 
Clearly, the distributions of the null model peak at about 
zero, while the empirical distributions peak at a posi- 
tive value. In addition, the empirical distributions, as 
a whole, lie in the right of the distributions of the null 
model. From the cumulative distributions, one could see 
that for the null model, less than 50% of users are of 
positive i?, while for the empirical data, more than 70% 
of users are of positive R. Aforementioned comparison 
shows the significance of the anchoring bias in empirical 
data. 
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FIG. 4; (Color online) Distributions of users' Pearson corre- 
lation coefficients R. (a) and (c) are for MovieLens, while (b) 
and (d) are for WikiLens. (a) and (b) show the histograms 
where p[R) is the probability density of R, while (c) and (d) 
show the cumulative distributions where P{R) denotes the 
fraction of users whose Pearson correlation coefficients are 
less than R. In each panel, results from the real data and the 
null model are respectively colored in red and blue. 





A 

^ 0.04- 

02- Slope=-0.038 \ 

^ )K RealC 

0.00- Linear Fit 



-0.04 





WikiLens | 








(d) 


Slope=-0.038\ 




^ Real Data 




Linear Fit 





10" 



10° 

L 



10 = 



1tf° 



10°° 10°' 



10'° 

L 



FIG. 5: (Color online) The average Pearson correlation co- 
efficient R[L) versus the correlation length L for MovieLens 
(a) and WikiLens (b). The blue and black curves respectively 
represent the results of the real data and the null model. Pan- 
els (c) and (d) display how the difference between R[L) of the 
real data and the null model changes with L in the linear-log 
scale. 



If the number of votes of a user i is larger than L, 
we could extend the Pearson correlation coefficient Ri 
to an L-dependent coefficient Ri{L) as the Pearson cor- 
relation coefficient of two series r^Oj, rio^^ ■ ■ ■ jfiOk -l 
and r-iOi+i, ''iOi+j: • • • j'^iOk, • As shown in figure 5, the 
average value of the L-dependent Pearson correlation co- 
efficient, {R{L)), of the empirical data over all users is 
remarkably larger than that of the null model for small 
L, and the difference between {R{L)) of the empirical 
data and the null model decays in a logarithmic way as 



A{R{L)) ^ A-BlogL, 



(8) 



where A sa 0.08 and B « 0.04 for both MovieLens and 
WikiLens. This result again indicates the existence of 
the anchoring bias. Moreover, it suggests that this bias 
will last a considerable time period, which is in accor- 
dance with previous experimental results on the duration 
of anchoring effects [30| . 

Combining those aforementioned experiments, the ex- 
istence and significance of the anchoring bias in online 
voting is obviously validated, whose pattern, as shown in 
figure 3, is very similar to the memory-embedded time 
series |29| . The extent of the anchoring bias, quantified 
by the difference of the average regulated rating from the 
null model, decays in a logarithmic form and will last a 
considerable duration. Most known literature on anchor- 
ing effects considered people's judgements, evaluations, 
estimations and predictions in offline world, meanwhile 
the quick development of Internet and the data process- 
ing technologies allow us to study the rich social psycho- 
logical phenomena in online world. Quantitative analy- 
sis and statistical description based on BIG DATA may 
build up a new paradigm for social psychology and facil- 
itate the birth of a new branch of psychology, probably 
called Internet psychology. This work is an elementary 
attempt that tries to uncover underlying decision-making 
processes based on extensive statistical analysis. Our 
findings arc helpful in understanding the online voting 
pattern and improving the performance of rccommendcr 
systems. 



Acknowledgments 

This work is partially supported by the Fundamental 
Research Funds for the Central Universities of China and 
the National Natural Science Foundation of China under 
Grant Nos. 11222543 and 11105024. 



[1] A.-L. Barabasi, Nature (London) 435, 207 (2005). 

[2] A. Vazquez, J. G. Oliveira, Z. Dezso, K.-I. Goh, I. Kon- 
dor, and A.-L. Barabasi, Phys. Rev. E 73, 036127 (2006). 

[3] D. Brockman, L. Hufnagel, and T. Geisel, Nature (Lon- 
don) 439, 462 (2006). 



[4] M. C. Gonzalez, C. A. Hidalgo, and A.-L. Barabasi, Na- 
ture (London) 453, 779 (2008). 

[5] M.-S. Shang, L. Lii, Y.-C.Zhang, and T. Zhou, EPL 90, 
48006 (2010). 

[6] C.-J. Zhang and A. Zeng, Physica A 391, 1822 (2012). 



5 



[7] J. Vig, S. Sen, and J. Riedl, Proceedings of the 16th [18 
International Conference on Intelhgent User Interfaces, 
p. 93-102 (ACM Press, New York, 2011). [19 
[8] L. Chen and P. Pu, User Modelingand User- Adapted In- 
teraction 22, 125 (2012). 
[9] L. Lii, M. Medo, C. H. Yeung, Y.-C. Zhang, Z.-K. Zhang, [20 
and T. Zhou, Phys. Rep. (in press). 
[10] Y. Koren, Commun. ACM 53(4), 89 (2010). [21 
[11] Y. Koren and J. Sill, Proceedings of the 5th ACM Inter- 
national Conference on Recommender Systems, p. 117- [22 
124 (ACM Press, New York, 2011). 
[12] N. Alon, M. Babaioff, R. Karidi, R. Lavi, and M. Tennen- [23 
holtz. Proceedings of the 13th ACM International Con- 
ference on Electronic Commerce (ACM Press, New York, [24 
2012). 

[13] J. Huang, X.-Q. Cheng, H.-W. Shen, T. Zhou, and X. Jin, 

Proceedings of the 5th ACM International Conference on [25 
Web Search and Data Mining, p. 573-582 (ACM Press, 
New York, 2012). [26 

[14] I. Simonson and A. Drolet, J. Consumer Res. 31, 681 

(2004). [27 

[15] C.-S. Wu, F.-F. Cheng, and D. C. Yen, Behaviour & 

Information Technology 31, 829 (2012). [28 

[16] G. Ku, A. D. Galinsky, and J. K. Mumighan, J. Person- 
ality and Social Psychology 90, 975 (2006). [29 

[17] A. Beggs and K. Graddy, Am. Economic Rev. 99, 1027 [30 
(2009). 



W. A. Hunt and W. College, Am. J. Psychology 54, 395 
(1941). 

T. J. Thorsteinson, J. Breier, A. Atwell, C. Hamilton, 
and M. Privette, Organizational Behavior and Human 
Decision Processes 107, 29 (2008). 

M. Kaustia, E. Alho and V. Puttonen, Financial Man- 
agement 37, 391 (2008). 

L. Cen, G. Hilary, K. C. J. Wei, and J. Zhang, Social 

Science Research Network no. 1572855 (2010). 

A. Furnham and H. C. Boo, J. Socio-Economics 40, 35 

(2011). 

J. Laherrere and D. Sornette, Eur. Phys. J. B 2, 525 
(1998). 

T. Zhou, B.-H. Wang, Y.-D. Jin, D.-R. He, P.-P. Zhang, 
Y. He, B.-B. Su, K. Chen, Z.-Z. Zhang, and J.-G. Liu, 
Int. J. Mod. Phys. C 18, 297 (2007). 
A. Clauset, C. Shalizi, and M. E. J. Newman, SIAM Rev. 
51, 661 (2009). 

M. E. J. Newman, Proc. Natl. Acad. Sci. U.S.A. 98, 404 
(2001). 

X.-Z. Ren, Z. Yang, B.-H. Wang, and T. Zhou, Chin. 
Phys. Lett. 29, 038904 (2012). 

R. Lambiotte and M. Ausloos, Phys. Rev. E 72, 066107 
(2005). 

K.-I. Goh and A.-L. Barabasi, EPL 81, 48002 (2008). 
T. Mussweiler, Eur. J. Social Psychology 31, 431 (2001). 



